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Abstract 


A design and application of adaptive online exam system are carried out in this paper. Adaptive exam systems determine 
different question sets automatically and interactively for each student and measure their competence on a certain area of 
discipline instead of comparing their gains with each other. Through an adaptive exam technique, a student’s distraction and 
motivation loss that is led by the questions with quite lower hardness level than his/her competency is prevented. In addition, 
negative effects of questions requiring higher knowledge than his/her competency over a student’s self confidence and morale are 
dismissed. Since questions are specialized so that they can allow making clear deductions about student gains, they are able to 
detect student competencies more effectively. Requiring less total time for measuring and being more flexible in the exam 
management are among the advantages provided by the system. Self sufficiency of the system in terms of planning, repeating and 
assessment of the measurement process especially allows itself to be used in the individual education sets. Through this system, 
student competencies can be determined more effectively in cases such as distant-learning, in which some challenges are 
experienced frequently. 
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1. Introduction 


Almost all of the contemporary testing systems are applied as paper-based. There have been numerous studies to 
apply exams online and carry out the measurement and evaluation processes through computers since technological 
means such as computer and internet have been widely used in educational activities. In these exams, students take 
tests either on a printed document and place their answers on a sheet; or they answer them on a computer screen. 
Besides, in these sorts of exams, all students are required to answer the same questions. 

Particularly during recent years, artificial intelligence applications toward comprehending human thinking 
process and copying this structure within the computer environment have been more prominent. The fuzzy logic, 
which is taken as an artificial intelligence at the same time and which also enables modeling of human behaviors 
and even expressing the undetermined concepts in mathematical form, has been a method that is applied frequently 
with regard to the resolution of decision making problems. Education is one of the main fields where fuzzy logic is 
utilized. (Kinshuk et al., 2001). Within the education process, a series of incidents that require decision making can 
be encountered. Especially, during measurement and evaluation process, it is a challenging process which requires a 
great deal of experiences to make the right decision and evaluate students properly. The issue of evaluating students’ 
performance without any mistakes has a structure to be modeled through fuzzy logic method because of 
indefiniteness and subjective scales contained (Karal & Cebi, 2011). 

In this study, the exam system and design stages designed by means of test algorithm in the adaptive exam 
system below are explained. 
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A facility value is assigned to the each question according to a determined fixed scale (e.g. 0-100 or 0-10 or 0-1 
etc.). For the question selection, binary-tree technique algorithm is employed. 

This algorithm works in the way below: 

1- Among the questions in the relevant subject, a question with medium level difficulty is asked first (e.g. if 
range of 0-100 is used, a question with 50 difficulty score). If student answers the question correctly, higher 
difficulty bracket (50-100 range) is halved to determine the difficulty level of the following question; in our process 
the question with level of 75 is asked. If the student answers the question erroneously, the lower difficulty bracket 
(0-50 range) is halved and a question with 25 difficulty level is asked. 

2- In the successive questions, the difficulty score of the next question is determined by halving the upscale or 
downscale threshold value range due to the difficulty level of the latest question. 

3- There are essential threshold values required to be used in the algorithm such as: 

eThe maximum amount of question, 

¢ The minimum amount of question, 

* The largest difficulty score difference to let us consider the swinging has slowed down, 

¢ The number of question (usually calculated as a function of standard deviation) needed to be asked to students 
to finish the test after the slowdown of the swinging (within the same swinging), 

¢ The interpolation strategy followed after the algorithm stopped (directly related to the threshold specified 
above). 

4- There must be sufficient amount of questions (on the same difficulty level) prepared to reduce the probability 
to face the same question to the acceptable level. 


2. Expert systems 


The expert systems, searching for how well the computers get human conducted jobs done, is one of sub- 
branches of artificial intelligence programming science . An expert system is a computer software relying on 
knowledge and deduction to perform a hard task which is supposed to be undertaken by expert people. It has a 
database consisting of information specialized in an expertise subject as it is similar to the competency of a person 
in a specialization area such as mathematics. Expert people obtain results by making logical deductions based on 
their knowledge in their specific area of expertise. Again, expert systems make deductions and reach results based 
on their information (Onder, 2003). 

Expert system; is a computer software that can perform job in a complex system conducted by an expert person. 
These systems also can be referred as computer software with consultancy quality which can perform expertise jobs 
requiring analysis, classification and diagnosis (Allahverdi, 2002). 

Expert systems can handle irresolute problems by providing expert support and can bring solutions. Due to their 
structures, database interface and decision support systems can be employed at the same time (Onder, 2003). 


3. Characteristics of the adaptive exam system 


Received answers as a result of an adaptive test application can be used in determining the next question or the 
question set. While certain amount of correct answers lead to the selection of questions with equal or higher level of 
difficulties; similarly, certain quantities of erroneous answers lead to the selection of questions requiring less 
competence. Thus, competence and developmental levels of individuals can be determined with fewer questions. 
Therefore, by individualizing exams, students are not required to complete whole question sets which are not 
suitable to their development level (Wise and Plake, 1987). 

Adaptive web-based exam is the process of matching up of measured individual competence (talent) and item 
difficulty in the computer environment (Hambleton et al., 1991). In other words, the application is based on 
adjusting a test to the individuals’ measured competence level; and on application of different sets of test items due 
to an individual’s position in his/her measured characteristics (Weiss, 1988). In this case, each individual receives 
different tests with different difficulty levels adjusted to their competence level. Conventional tests are usually 
arranged to measure mean competence level. Adaptive test application provides an environment to reveal the best 
measurement of an individual’s competence level. A general adaptive test application follows the steps below (Lord 
and Stocking, 1988): 

1- Competence level of an individual is estimated. 

2- Items which can be good measures of that individual’s competence are selected from the pool consisting of 
related parameters. 
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3- The selected items are applied and scored, and the estimated competence level is reviewed. 

4- If the error in the competence estimation has reached sufficiently low level, the application is terminated; if 
not, turned back to the second step. 

Exams requiring high level of security must be performed in secure exam rooms under surveillance such as 
computer lab. Many specialized test distribution method can be supported by internet today. Web-based exams 
allow test distribution and security subjects in the specially determined test centers. 


4. Enhanced web-based adaptive exam system 


The developed adaptive exam system provides a wide variety of applications to students and lecturers due to its 
design and simple visual characteristics which do not require users to have any further guidance. By means of 
different methods, exams can be implemented, analyzed; and measurement and evaluation processes can be 
performed. In addition to the application of exams through an adaptive way, student monitoring and guidance as 
well as assessment of exam results, which is normally neglected due to its time-consuming structure but important in 
terms of achieving educational targets, can be more convenient. 


4.1. General structure of the system 


The system is composed of three-layered structures: a database for data storage, a server for application and 
clients which are connecting to the application server. As database, MYSQL is used since; 

1- MySQL database management system is open-coded software and it has enhanced specifications, 

2- It uses application server layer, 

3- On the client layer side, it is required to employ a web browser (Internet Explorer, Mozilla, Opera etc.). 

To develop Online Exam system, a server based and fast PHP programming language is preferred. 

For the developed software, Apache is used as a web server which is a strong, skilled and flexible HTTP (Hyper 
Text Transfer Protocol) server and open-coded programming. The web server is a software sending the pages stored 
under the web address you are connected to (Netcraft, 2012). 

Macromedia Dreamweaver 8 software is used to prepare the interface of the system. Dreamweaver is one of the 
most preferred HTML editors due to its enhanced properties and conveniences; and allows server-based languages 
to be used to create dynamic pages. 

In the adaptive web-based exam system, JavaScript language is used to allow dynamic user access; to let it 
probed in the same page; and to perform tasks such as presenting resting time for the exam. 


4.2. Security 


To ensure data security, IPSec or Kerberos must be used between database and application server. Since IPSec 
provide secure communication between database and application server, it prevents third parties intrusion into the 
database. In addition, SSL must be used to assure secure communication between the client and the server. In this 
application, securing passwords of administrators and lecturers is of crucial importance. It is quite simple for the 
third parties to capture user names and passwords sent over the network without any encryption. Therefore, the data 
sent over the network must be encrypted with SSL. Furthermore, as it has been presented in the previous sections, 
passwords must be stored as encrypted in database and adequate precautions must be taken against SQL injection 
attacks (Akin, 2007). 

Some users who are searching for an opportunity attack the system may wish to access and harm the system 
through SQL injection commands. In the developed application, PHP and MYSQL commands, such as 
HTMLENTITIES, MYSQL _ REAL ESCAPE STRING, STRIP_TAGS, are used to prevent unauthorized entries 
into the system and precautions are taken against those attacks. Additionally, in the inquiries requiring user name 
and password, encryption algorithms such as Md5 and SHA1 are used. 

When users are granted access into the system, user names, session information are kept and these are checked in 
every page requiring access to prevent unauthorized access. Besides these security precautions, users must maintain 
their network security at the highest level by means of antivirus and anti trojan programs. 
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4.3. User groups 


There are three types of users in the system: system administrator, lecturers and students. System administrator 
has the authorization to add new users, courses into the system, assign new lecturers and students to courses, and 
add announcements. Lecturers are authorized to define classroom, courses and students, connect the classroom and 
the course, add and modify the questions, change student passwords, edit questions, access exam results, include 
classroom lists, transform exam results into the excel form for further analysis, and send messages to parties. When 
students access into the system, they see the exams they are required to take; they can take these exams; and they 
can view their exam scores. User types and roles in this application are presented in the Table 1. 


Table 1. User Groups 


User Access Authorization Limitations 
Administrator To the administrator page Can view all data in the system. All rights granted. 
echirer: To the lester pane: Can view and amend all information Cannot access to the courses given by the other 
pag regarding given courses. lecturers. 
Caitake tis esmne Obihe Biven eoutees Can only access to the exam information defined for 
Student To the student page 8 themselves and to the general — student 


and view data about them. 
announcements. 


4.4. Organization and setting 


In the web-based exam system, the user name and the password are the keys of the system to identify a specific 
user. Since any person with the user name and the password is considered to be authorized to access into the system, 
the information security is critical. The administrator can automatically generate the user name and the password for 
students registered for the course and for the lecturers of the described course; and send this information to parties in 
a secure way. 


5. Definition and implementation of the exam 


An exam can be organized by the lecturers of the course from the question pool that they have already created or 
from the previous questions entered into system database. Exams can be prepared in advance but at the same time 
students’ access are not allowed. To that end, the exam is performed under IP limitation so that it can be 
implemented only in specific computers or laboratories. 

Texts, pictures and animations can be used in created questions. The system can be easily accessible twenty -four 
seven from any internet access point. Since all information regarding the exam session is transferred to database 
through the web interface, lecturers can also reach students’ data through web interface. 

The web interface used for the exam system is designed in a way to ensure security by using the least 
information level and most user-friendly form. In this application, each exam result can be updated along with the 
new information defined. A user manual for the interface applications is a preferred characteristic of the system. 


5.1. Creating an exam 


Exams can be created by two different methods. In the first method, lecturers can create an exam by selecting as 
many as questions they need with a certain difficulty level from the question pool that they previously created by 
entering questions with different difficulty levels. While the exam is being created, which group to take it is 
predetermined. Thus, students view the exam created for them when they access into the system. On the other hand, 
in the second method, the lecturer lists all questions recorded in the question pool for the relevant exam, then selects 
the desired questions among them. 


5.2. Exam data 
For automated evaluation of exams by the system itself, it is also possible that raw data generation process for 


analyzing can be performed by using obtained information and exam data. The data are evaluated through statistical 
methods and assessments can be made about the current exam system as well as they can constitute resource for 
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future studies. Above all, the selection processes of most suitable questions and dismissal of questions not 
accommodating validity, reliability and other analysis criteria from the question pool can be more convenient. 


6. Database design 


The database infrastructure is carefully emphasized in terms of circulation and recording processes of data 
through the web-based exam system. The database is designed to compile the largest amount of data by using 
minimum amount of system source while it is keeping all data safe. 

Main tables in the system are Courses, Questions, Exams, Students, Lecturers, Administrators, Semesters, 
Majors, and Results. Additionally, there are auxiliary tables of Lecturer Course, Student Course, 
Semester Courses, Exam_Questions, Student_Answers. 


7. Results 


Within the developed exam system, an exam can be implemented by means of both the conventional method and 
the adaptive exam method. In this study, design and application stages of the exam system are carried out. 
Furthermore, the designed exam system was implemented in the midterm and final exams through the conventional 
method for the students taking the course of Computer I at the Faculty of Education, Ahi Evran University (Kirsehir, 
TURKEY) in spring semester of 2011-2012 academic year. 

Framework for the academic success is comprised of 65 items. To enhance the content validity of the test 
coverage, proportion of the subjects in the distribution of questions is taken into consideration. Test validity is 
provided based on experts’ views. Validity of the tests is ensured based on the pilot study applied to 90 students in 
two different classes. As a result of the application, 15 items whose distinctiveness strength is below 0.30 are 
dismissed from the academic success test. While the distinctiveness strength of the rest 50 items is in the range of 
0.30 and 0.95; their item difficulty varies between 0.27 and 0.64. As a result of the calculations, the mean difficulty 
level was 0.52; and content consistency coefficient was (Kr-20) 0.88, and their standard deviation was 4.5. Thus, 
difficulty levels and distinctiveness values required for the question bank, essential part of the adaptive exam 
system, are determined. 

In the previous studies it was concluded that “It is presented that student performance evaluation problems can be 
solved by means of multi-criterion decision making methods; and similar methods can be employed in different 
measurement and evaluation applications” (Karal and Cebi, 2011). 

In further studies, practicability, validity, reliability, advantages and disadvantages of adaptive exam systems will 
be discussed through the statistical analyses over data that will be obtained as a result of the future adaptive exam 
system applications. 
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